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(54) Foreign object video detection and alert system and method 



(57) A foreign object video detection system com- 
prises a television camera for producing a digital color 
image of a work surface, a converter having direct mem- 
ory access to a computer, color detection and color 
image processing software, logic for discriminating 
objects deemed to be a foreign object on the work sur- 



face or upon a layer of material on the work surface, 
togic for providing appropriate warning to the operator, 
and input controls for selecting the area of interest for 
detection and for optimizing the detection technique 
based upon the manufacturing situatioa 
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Description 

FIELD OF THE INVENTION 

[0001] The present invention relates generally to 
methods and apparatus for video detection of foreign 
objects on a work surface arxl warning the operator by 
computer display representation, warning light, vibrat- 
ing beeper, and/or audio warning during manufacture of 
composite structures containing stacked layers of mate- 
rial. 

BACKGROUND OF THE INVENTION 

[0002] In the aerospace industry, composite struc- 
tures are often chosen instead of metallic structures 
when high strength to weight ratio is desired, especially 
for complex shapes. Strength in such composite struc- 
tures is greatly reduced, however, when defects occur, 
such as delaminatfons. The introduction of a foreign 
object within a stack of composite plies comprising a 
composite structure can cause such strength-redudng 
delaminations In a safety crittoal application, such as 
the composite sWn of a tactical aircraft wing, such a 
reduction in strength may exceed allowances, forcing 
expensive repair or scrapping of the composite struc- 
ture. 

[0003] Traditionally, manufacture of such composite 
structures has often not been automated due to the dif- 
ficult techniques employed and small numbers pro- 
duced. Each layer of material, or ply, is placed by hand. 
Thus, each composite structure may be in a partial state 
of assemtrfy for an extended period off time with many 
opportunities for human error in overlooking objects left 
within the composite plies. After the structure is com- 
pletely assemt>led and cured, nondestructive testing 
may be used to verify fabrication. See Figure 9. 
[0004] Although post-manufacturing nondestructive 
testing may be sufficient for preventing unsafe struc- 
tures from being used, the cost of repair or replacement 
Is significant if an embedded object is detected. An even 
more problematic result is when a strength critical struc- 
ture is flawed by a foreign object tiiat is not detected by 
post-manufacture testing, leading to catastrophic fail- 
ure. 

[0005] Although not utilized during conposite struc- 
ture fabrication, it is known in other industries to empkiy 
closed circuit television for process control where the 
parameter of interest is an edge or boundary which may 
be compared with a pre-existing reference. For exam- 
pie. US. Patent No. 3.243.509 to Hans Sut discloses a 
system which employs a TV camera to detect the phase 
boundary between the solid and liquid phases of a sem- 
iconductor rod In a zone melting process. In U.S. Patent 
No. 4.064,534 to Tung Chang Chen et ai., a TV camera 
is employed as part of a quality control system in the 
manufacture of glass bottles with tfie outiine of the fin- 
ished botties being compared with that of a reference 



INSOOCID: <EP 0903574A2 I > 



bottie. In US. Patent Na 4.135.204 to Ray E. Davte. Jr. 
et ai.. a TV camera is used to control the growth of a 
thermometer end opening Ulster in a heated hollow 
gfass rod by monitoring and Iteratively controlling the 
5 growth of the edges off the t)lister using edge detection 
techniques. In US. Patent Na 4.760.444 to Paul 
Nielsen et al.. the inspected area had to remain in a very 
specific location so that a scanned image coukJ be 
directiy compared wrtfi the known light levels off the 
10 inspected area in the desired state. 

[0006] It is also known to employ a dosed circuit tele- 
vision camera in a process for peribrming pattern recog- 
nition and area measurement applications. In U.S. 
Patent No. 4,445.185 to Ray E. Davis. Jr.. et al., the dig- 
15 m video image is p>rocessed in a DRAM of a computer 
by comparing the subject image with a stored master 
image in order to determine if a threshoW difference is 
reached. The subject images are thus inspected for 
ffaws by determining whether they exceed this thresh- 
20 Old. In U.S. Patent No. 5.187.573 to Hajime Yoshida. a 
fransparent or translucent article is back lit and the 
resulting image is subjected to a pattern recognition 
techrtque for expected flaw, US. Patent No. 5.452.370 
to YasuyuW Nagata is an example of robotic vision 
25 measuring of articles and peribrming calculations refat- 
ing to the object prior to manipulation. It is also known to 
use color in detecting foreign objects. In US. Patent No. 
4.120,402 to Ronald Swanson, defective tomatoes or 
foreign objects are detected by physically placing the 
30 otjject in tiie correct place for inspection and detecting 
specific colors for discriminating culls or agricultural 
debris. 

[0007] These prior art systems are directed to situa- 
tions in which manufacturing is automated and the 
35 desired state is known. These systems cannot adapt to 
changes In the process being empfoyed by the operator 
or adequately interface with the operator As such, con- 
ventional ctosed circuit televisions have not be&} capa- 
ble of effectively monitoring conposite structure 
40 fabrication. 

SUMMARY OF THE INVENTION 

[0008] It is therefore an object of the present Invention 
45 to provide video detection of foreign objects and give 
warning to an operator. 

[0009] It is another object of the present invention that 
such warning to the operator Include an audio warning, 
a warning light, beeper vibration, and/or a dispfay of the 
50 work surface with each detected foreign object (FO) 
highUghted. 

[0010] It is an additional object of the present Inven- 
tion to include several color image processing alterna- 
tives for discrimination a foreign object while reducing 
55 false afarms. These alternatives include detecting 
known unacceptable colors, detecting colors on a prep- 
aration table not known as acceptable, and detecting a 
color not present in a prior image. 
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play generated by the color image processing sys- 
tem showing a lay-up table after the color image 
processing system subtracts areas not of concern 
for foreign objects. 

5 

Figure 5 is color image 4 of 4 of an illustrative dis- 
play generated by the color image processing sys- 
tem after detecting a foreign object by Hs color. 

10 Figures 6A and 6B are top and side views of a lay- 
up table. 

Rgure 7 is a perspective view of the lay-up table. 

15 Rgure 8 is a flew diagram for the method of video 
detection of foreign objects 

Rgure 9 is a prior art process for constructing a 
composite structure incorporating post-manufactur- 
20 ing detection of foreign objects. 

Rgure 10 is a new process for constructing a com- 
posite structure incorporating concurrent detection 
of foreign objects with manufacture. 

25 

Rgure 11 is a block schematic of an embodiment of 
the video detection apparatus for identifying foreign 
otijects. 



[001 1 ] It is yet another otaject of the present invention 
to provide video detection of foreign objects having at 
least one vkjeo camera oriented toward a work surCaca 
The numt>er. position. nrK)bility. and resolution capability 
of these cameras being selected based upon the size 
and contour of work surface and size of foreign objects 
of concern. 

[001 2] It is a further object of this present Inventicxi to 
provkJe a method for video detection of foreign objects 
arxl warning to operators as part of the manufacture of 
composite structures. 

[001 3] These and other objects are provided, accord- 
ing to the present invention. t>y processing the video 
color image of a work surface within a defined area of 
interest, detecting specific colors associated with for- 
eign objects or colors that differ from the corresponding 
colors of a prior image of the work surface. If a foreign 
object Is detected, the operator is then warned by high- 
lighting the location of the foreign object upon a compu- 
ter display and/or by actuating a warning light, audio 
warning or electromagnetic signal to gain the attention 
of the operator. 

[0014] By detecting for^gn objects based on the color 
of the foreign objects, the detection method and appara- 
tus can detect foreign objects that may be of unpredict- 
able shape, color or location. In this regard, the present 
invention uses the discriminating potential of tiie differ- 
ent colors of potential foreign objects to optimize the 
detection algorithms. Advantageously, tiie apparatus of 
the present invention does not require the work piece or 30 
the camera to be physically moved in order to define the 
area for conparison. Even if used in its most basic fash- 
ion of looking only for foreign objects of predetermined 
colors, the present invention incorporates a sophisti- 
cated operator interface for more flexibly monitoring the 3s 
process and for providing options for warning the oper- 
ator. 

BRIEF DESCRIPTDN OF THE DRAWINGS 

40 

[0015] 

Figure 1 is an example of incorporating the foreign 
object video detection method and apparatus of the 
present invention into an existing apparatus for 4s 
manufacturing composite structures. 

Figure 2 is color image 1 of 4 of an illustrative dis- 
play generated by the color image processing sys- 
tem showing a lay-up table before adding a layer of so 
material or a foreign object. 

Rgure 3 is color image 2 of 4 of an illustrative dis- 
play generated by the color image processing sys- 
tem showing a lay-up tat3le after the introduction of ss 
a layer of material and foreign objects. 

Figure 4 is color image 3 of 4 of an illustrative dis- 



DETAIl£D DESCRIPTION OF THE INVENTION 

[0016] The present invention now will be described 
more fully hereinafter with reference to the accompany- 
ing drawings, in whk:h a preferred embodiment of the 
invention Is shown. This invention may, however, be 
emtxxfied in many different forms and should not be 
construed as limited to the embodiments set forth 
herein; rather, this embodiment is provided so that ttiis 
disclosure will be thorough and complete, and will fully 
convey tiie scope of tfie invention to those skilled in tiie 
art. Like numbers refer to like elements tfiroughout 
[001 7] Material inadvertentiy introduced into a product 
or process is described as a foreign object, or FO. Often 
in tiie aerospace industry, It may also be termed foreign 
object damage, or FOD. emphasizing its result. 
[001 8J Although tiie video detection and alert system 
and method can be implemented in a variety of man- 
ners, the video detection and alert system and method 
of the present invention can be advantageously inte- 
grated into an existing manufacturing process with 
video cameras overlooking the work surfeices as shown 
in Rgure 1. 

[001 9] The inventive nature of the detection portion of 
the system, as opposed to the warning portion, is best 
illustrated by a simplified depiction of tiie processed 
images. As illustrated in Figure 6 and Figure 7. the work 
surface 24 may have an outermost guide 23 and cutout 
guide 22 for con-ectiy positioning the layers of material. 
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or composite pli^ The video detection and aJert sys- 
tem includes a camera having a field of view 20 which 
does not necessarily need to be coextensive with the 
space defined by the outermost guide 23 in which the 
layers of material wiD be placed. The camera generates 
video images, such as the video digital image shown in 
Rgure 2 of the table prior to placement of material or a 
foreign object The video digital image 30 con'esponds 
to the field of view 20 of the video camera, and includes 
an area of interest 31, preferably corresponcfing to the 
shape defined by the outermost guide 23. As described 
below, the area of interest is typically defined by opera- 
tor input Although not shown in Rgure 2, the video dig- 
ital image could include color or contrast changes, such 
as glare 32 from the illuminating light source or color dif- 
ferences between the cutout guide 22 and the work sur- 
face 30 and a nonhomogeneous portion 33 of the work 
surface 30. 

[0020] In Figure 3, the area of interest 31 of the video 
digital image 30 generated by the camera has been 
changed by the addition of a layer of material 35. a non- 
problematic foreign object 37 outside of the area of 
interest 31. and a problematic foreign object 39 within 
the area of interest 31. As described below, the video 
detection and alert system includes a color image 
processing system for processing the video digital 
images generated by the camera. In this regard, the 
cofor video processing system initially subtracts or 
removes that portion of the video digital image 30 which 
is either outside of the area of interest 31 or is inside the 
area of interest but is othenwise predetermined to not 
be of interest thereby creating the video digital image 
shown in Figure 4. for example. In this instance, the 
video processing system removes those areas between 
the outer boundary of the video digital image 30 and 
area of interest 31 as well as an inner area defined by 
the cutout guide 34 which becomes a featureless por- 
tion 38. Note that the nonproblematic foreign object 37 
is removed from further consideration since it lies out- 
side of the area of interest. Some objects such as the 
glare 32 that should be ignored are only partially 
excluded at this point 

[0021 ] Once those portions of the video digital image 
30 which are either outside of the area of interest 31 or 
are inside the area of interest, but are othenvise prede- 
termined to not be of interest, have been removed, the 
video signal processing system 47 identifies all objects 
having one or more predefined colors. Typically, the pre- 
defined colors have been identified In advance by the 
operator or the system designer to include the colors of 
the most common types of foreign objects. However, the 
predefined colors preferatsly do not include the colors of 
the composite plies or the work tat>le. By identifying all 
objects having one of the predefined colors, the video 
processing system 47 will therefore, in all probability, 
identify all foreign objects within the area of interest 31 . 
[0022] Alternatively, the operator can identify those 
cofors which should not be detected by the video 



processing system 47. such as the colors of the com- 
posite plies or the work tat>le. As such, the video 
processing system of this embodiment will identify all 
objects having a color other than those specific cotors 

5 identified t>y the operator. 

[0023] According to either embodiment, an image of 
the area of interest can then l>e displayed to the opera- 
tor. During this subsequent (fisplay to the operator, the 
warning system of the foreign object detection system 

10 may display the raw video digital image 30 shown in Rg- 
ure 3 with those features having an unanticipated color, 
such as foreign object 39. highlighted to thereby draw 
attention to the problematic foreign object 39. The over- 
all foreign object detection and warning method is illus- 

is trated in Rgure 8. 

[0024] As illustrated in Rgure 9. a conventional proc- 
ess for buikling the oomposite skin for tactical aircraft 
wing structures begins by laying ttie ply on the prepara- 
tion table. Typrcally. two ac^acent tables are used in pre- 

20 paring and laying up the layers of material, or composite 
plies. The preparation table is a typical, work table upon 
which a layer of composite material is lakl. The protec- 
tive backing is then peeled off the exposed face of the 
composite layer arxJ discarded. Then, ttie layer of com- 

25 posite material is placed upon the lay-up table wittiin the 
tooling or guides. The side previous^ exposed from 
which the protective backing has been removed is now 
laid foce down upon the lay-up table. Since the protec- 
tive tracking has been removed from the downwardly 
30 facing side of the composite layer, tiie composite layer 
adheres to the underlying composite layer. The other 
side of the oomposite layer is now exposed arxJ has its 
protective backing peeled off and discarded in prepara- 
tion for another composite layer to t>e placed tiiereupon. 
35 Once the entire composite structure has been laid up 
and cured, the resulting composite structure is sub- 
jected to non-destructive testing to detect flaws and/or 
foreign objects witNn the othenAfise completed struc- 
ture. This last step of post-manufacture non-destructive 
40 testing emphasizes the reactive rather then proactive 
controls for foreign objects practiced by conventional 
fabrication processes. 

[0025] As illustrated in Rgure 10. the foreign object 
detection and warning system and method monitors 

45 these work surfaces as the composite plies are laid up 
to detect foreign objects prior to completing the com- 
posite fabrication process. Altiiough ttie foreign object 
video detection and warning system can monitor work 
surfaces of any size, the foreign object video detection 

50 and warning system of one embodiment continuously 
monitors a 5 ft by 12 ft work surface in demonstrated to 
detect one-quarter inch square foreign objects. It is 
anticipated that production versions could consistently 
detect one-half inch square objects. If ttie foreign object 

55 video detection and warning system detects a foreign 
object at anytime during the fobricating process, ttie 
operator is warned and can take proactive st^s to 
remove the foreign object and rework the part prior to 
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completely assembling and curing the part Conven- 
tional post-manufacture non-destructive testing can still 
be employed in conjunction with the video detection arxj 
warning system to detect other types of defects such as 
material flaws, ooniposite curing discrepancies, or han- 
dling errors. 

[0026] The present invention can ^Ivantageously be 
included into an existing manufacturing environment at 
relatively tow cost For example, the cost of one embod- 
iment of the foreign ot>ject video detection and warning 
system was approximately the same as the cost of one 
manufactured article that would otherwise have been 
scrapped for including foreign objects. The cost is even 
lower if a general purpose computer is available at the 
location that can be time shared with this monitoring 
process or if multiple woric stations can utilize the same 
computer. 

[0027] Another advantage of the video detection and 
warning system is that tiie cameras need not be physi- 
cally repositioned to account for changes in the area of 
concern. Instead, the color video image processing sys- 
tem can select the particular area of interest from the 
larger video image based upon input by the opa^tor. 
Althou^ some minimum lev^ of fighting is needed as 
desaOsed below, the work surfaces advantageously 
need not be specifically illuminated to obtain appropri- 
ate color images. Since the video detection arKi warning 
system detects foreign objects based upon color, the 
image processing system need not be programmed 
with a master image to which the current image is com- 
pared, thereby simplifying the design of the video detec- 
tion and warning system. 

[0028] As illustrated in Figure 1 1 . one embodiment of 
the foreign object video detection and warning system 
was demonstrated using off-tiie-shelf, commercial hard- 
ware with modified commercially available software. 
The foreign object video detection and warning system 
includes an NTSC video camera 41 attached to an 
existing frame structure shown in Figure 1, approxi- 
mately ten feet aiDove tiie preparation tat)le. The foreign 
object video detection and warning system also 
includes another video camera 41 similariy placed 
above the lay-up tatDle. As an alternative to NTSC video 
cameras, the foreign object video detection and warning 
system can include digital cameras if so desired. Based 
upon tfie color and contour of the underlying table, the 
detection technique can be correspondingly modified. 
For example, a simpler color detection algorithm may be 
appropriate for a preparation table presenting a uniform 
background. However, tiie lay-up table has a numt>er of 
guides for positioning the layers of material, or compos- 
ite ptieSi so as to disadvantageously create varying 
colors, glare and a contoured work surface, thereby 
requiring a more sophisticated detection algorithm. 
[0029] The video cameras of one advantageous 
emtxxliment capture and transmit standard television 
format video signals, via coaxial cable, to a converter 
45. in this instance a FRAME QRABBER™ converter by 



Premier Electronics Card Products of the United King- 
dom. The FRAME GRABBER™ converter of this 
embodiment includes a Microsoft WINDOWS"* compat- 
ible PCMCIA Type II video capture card which accepts 
5 the video signals and provides a corresponding digital 
color image to the video image processing system. In 
one emtxxliment the video image processing system 
includes an Intel PENTIUM™ microprocessor-based 
personal computer on which Microsoft VISUAL BASIC™ 
10 software has been installed for controlling Media Cyber- 
netics IMAGE PRO™ software. The IMAGE PRO™ soft- 
ware provides color image processing capability and 
color channel processing capabilities in order to sepa- 
rate objects, including toregn objects, from background 
IS based on color and to isolate a group of cotors for fur- 
tiier processing and analysis. The IMAGE PRO™ soft- 
ware also provides computer display representations of 
tiie digital color image for presentation upon an associ- 
ated display. The video processing system can also 
20 overiay tiie outline of the area of interest M X)n the cofor 
Image presented by the display 49 and can highlight for- 
eign objects by presenting the foreign objects in color or 
by othenvise designating tiie foreign objects, such as by 
superimposing a flashing red ellipse shape over ttie 
25 image of tiie foreign object on the color image display. 
The video image processing system and, in particular, 
tiie IMAGE PRO™ software permits tiie operator to 
select an area of interest ttiat is sut^sequentiy shown 
overlaid on the digital color image of the work surface. 
30 The foreign object video detection and warning system 
can include ottier types of warning devices in addition to 
highlighting the detected foreign object in tiie color 
image presented upon the display device 49. such as 
audible warnings or warning lights which are actuated 
35 by the video image processing system. The iteration 
rate of tiie foreign object video detection and warning 
system preferal)ly makes recognition and warning 
appear instantaneous for tiie operator. 
[0030] The foreign object detection and waming sys- 
40 tem performs better when the lighting is optimized since 
places on tiie work surface that return a reflection from 
tiie lighting may othenvise appear as a light colored for- 
eign ol3ject and produce a false alami. However, com- 
pensating for such glare can reduce the sensitivity of 
45 the detection system. Thus, the lighting can be opti- 
mized by utilizing diffuse lighting, controlling the direc- 
tion of the lighting, using glare redudng filters, and 
using spectral filters so that any reflection is not directed 
toward a video camera, and covering the t>ackground 
so surface with matte black material to minimize, if not 
eliminate, undesirable reflections, since the texture and 
color of the work surface should absorb rather than 
reflect light. The lighting slioukf also be sufficientiy 
bright for the sensitivity of the chosen video camera. In 
55 addition, extraneous sources of light can be shielded by 
enclosing portions of the working area. 
[0031 ] The foreign object detection and warning sys- 
tem also performs better when tiie work surface is made 
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as nearly homogeneous in texture and color as possi* 
t)te. especially if the work surtece can be made to be as 
dissimilar from all expected forms of foreign objects as 
possible. In one advantageous embodiment, the prepa- 
ration table was unifbrmly covered with a matte black 
paper used in the photographic industry comnwrdy 
called black backdrop. 

[0032] In order to increase the resolution or to obtain 
a larger viewing area, the foreign object video detection 
and warning system can include a camera designed 
and positioned as described below. Rrst the camera 
could be brought closer to the work surface. Second a 
higher resolutibn camera could be used. Third, a plural- 
ity of cameras could be used. Fourth, the camera posi- 
tion could be controllat^ly movable. Frfth, the direction of 
the field of view of the camera could be controllak^ly 
steered. Sixth, the camera could incorporate magnifica- 
tion lenses. 

[0033] It is also anticipated that the removal of 
detected foreign objects could be expedited by integrat- 
ing the foreign object video detection and warning sys- 
tem with an intense, nanrowly-focused directional light, 
such as an eye-safe laser 44 with a frequency in the vis- 
ual spectrum. The cotor image processing system 47 
could provide tfie coordinates of the detected foreign 
object 42 to the laser 44 which is thereafter oriented so 
as to emit a laser beam directed to the coordinate at 
which the foreign object is located upon the work table, 
such as by pointing at the center of the foreign object, 
defining tiie outiine of ttie foreign object, or illuminating 
the entire f(yeign object. The illumination of the foreign 
object can continue until all FO is found and removed. 
This technique is especially appropriate for manufactur- 
ing processes having laser pointers integrated with 
computer aided design or computer aided manufactur- 
ing CAD/CAM electronic design files capability. 
[0034] Although the above-described embodiment of 
the foreign object video detection and warning system 
converts the television video image to a digital color 
image, this conversion may not be necessary with other 
embodiments. For examples, the video camera could 
directiy produce a digital color image suitable for digital 
color image processing. Altematively, an analog cofor 
image processing system could directly process a tele- 
vision format video signal. 

[0035] Alternatively, the video camera couki be 
replaced with other optical scanning means. For exam- 
ple, the necessary color and/or contrast information 
coukl be gathered by scanning the work table with a sin- 
gle point detector. The pattern of scanning couW be ras- 
tered or specifically steered only over the area of 
interest The detector coukJ also be a single line scan 
camera. 

[0036] Although the foreign object video detection and 
warning system of the illustrated emtxxiiment includes 
an audio warning, a warning light and a vkJeo display 
warning the operator of the foreign object, the foreign 
object video detection and warning system could 



include a wkJe variety of other warning metiiods or 
devices. An alternative warning technique could include 
an electromagnetic signal generated by the conrputer. 
For exampla the computer coukJ contact a vibratnng 
5 beeper earned by the operator. 

[0037] The foreign object vkfeo detection and warning 
system also preferably optimizes the timing of the warn- 
ing. For exarriple. the foreign object vkleo detection and 
warning system could delay the warning in order to pro- 
10 vide the operator with the opportunity to remove a for- 
eign object before the foreign object detectfon system 
and method assumes that the operator has overlooked 
the foreign object In particular, an annoying audH> 
waming couU be delayed although other visual warn- 
is ings are not delayed. This selective delay of the various 
types of warnings wouki reduce nuisance alarms that 
ultimately degrade the utility of ttie system. Also, the 
color image processing system coukI provtie text based 
warning messages for operators likely to view a display 
20 screen. Moreover, automated systems for preparing 
and laying material couM incorporate the foreign object 
video detection and warning system such that ttie warn- 
ing signal interrupts an automated system controller, 
directs a rotXTtic manipulator to remove the foreign 
25 object, or transmits a signal to a human operator, per- 
haps at a distance, to intercede in correcting the prob- 
lem, such as by removing the foreign object prior to 
continuing the manufacturing process. 
[0038] Although not applicat>le for the types of mate- 
30 rial inspected in the above-desaibed embodiment the 
work table couU be bacMit. if so desired. Thin and trans- 
lucent materials wouki thus discfose foreign objects on 
top or under ttie layer of material due to the unk|ue color 
provkled by the foreign oksject or the reduced level of 
35 light transmitted theretiirough. By detecting the relative 
levels of light, the foreign object video detection and 
warning system of this entxxfiment couki also detect 
foreign objects that are the same color as the layer 
material, such as a scrap piece of tiie same material. 
40 [0039] In one advantageous embodiment, ttie foreign 
object vkieo detection and waming system receives 
operator input via a personal computer keyboard, a 
mouse or a hand held remote control to define the area 
of interest on the work surface for foreign object detec- 
ts tion. It is anticipated, however, that a computer aided 
design, computer akied manufacturing (CAD/CAM) sys- 
tem could be integrated with the manufacturing process 
and that the area of interest can be predefined via the 
CAD/CAM system. Alt^-natively. the physical limits of 
so the digital color image couM be constrained to be coex- 
tensive with the area of interest. 
[0040] Alttiough color image processing is described 
heretofore, the foreign object vkJeo detection and warn- 
ing system couki also distinguish foreign objects based 
55 upon gray scale image processing since most foreign 
ok)jects wouM differ in contrast from the composite plies. 
This implementation would be particulariy advanta- 
geous in low light situations in which a laser is used. 
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[0041 ] By utilizing the measurement capabilities of the 
foreign object video detection and warning system, the 
discrimination of foreign objects can be further 
enhanced. For example, the foreign object video detec- 
tion and warning system of the illustrated embodiment 
can provide the cartesian coordinates of the center of 
the object and information regarding the area of the for- 
eign object By detecting only those foreign objects hav- 
ing a size less than a predetermined value, the number 
of false alarms can therefore be reduced. For example, 
since relatively large, silver-colored tools are often used 
on the work surface, the system may be configured to 
only actuate an alarm for any foreign object equal to or 
larger than the size of the blade of an EXACTOR knife. 
[0042] By utilizing the area calculation and compari- 
son as described above, it is anticipated that the foreign 
object detection system and method could be applied to 
other situatfons. such as monitoring an aircraft parking 
area or manufacturing shop ftoor for foreign objects. 
Those objects deemed too large to be a foreign object 
wouki be ignored. Objects too small to be hazardous 
couU also be excluded by cafoulating their area, txjt 
woukJ probably be ignored anyway due to resolution 
limitations on the system. If the video camera is not per- 
pendicular to the area off interest, the foreign object 
vkleo detection and warning system could still calculate 
the size of a foreign abject by determining the range or 
distance from the camera to the foreign object, such as 
with a laser range finder, or by predefining a description 
of the environment For example, a look-up table of dis- 
tances from the camera to the foreign object couM be 
predetermined for the various angles of azimuth and 
direction. 

[0043] Furthermore, it is anticipated that the foreign 
object video detection and warning system coukf pro- 
vide cueing and activation commands to an automated 
foreign object removal device such as a robotic pick- 
andiDlace device, vacuum chuck, or air jet. 
[0044] In addition, the foreign object video detection 
and warning system could also incorporate a learning 
algorithm, such as a neural network, to assist in detect^ 
ing foreign objects. 

[0045] In the drawings and specification, there has 
been set forth a preferred embodiment of the invention, 
and. although specific terms are employed, the terms 
are used in a generic and descriptive sense only and 
not for purposes of limitation, the scope of the invention 
being set forth in the following claims. 

Claims 

1 . A detector for finding foreign objects lying on a work 
surface and providing warning, the detector com- 
prising: 

a video camera having a field of view, said 
video camera oriented such that the work sur- 
feice is in the field of view for producing a color 
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image of the work surface; 
a color image processing system, responsive 
to said video camera, for analyzing a prede- 
fined area of interest within the f iekJ of view of 
the video camera, sakj color image processing 
system comprising means for discriminating a 
foreign object within the area of interest of the 
color image based on visual attributes of the 
foreign object sakf color image processing 
system further comprising means for determin- 
ing a location of the foreign object on the work 
surface; and 

a warning mechanism, responsive to said color 
image processing system, for an alerting oper- 
ator when a foreign object is present on the 
work surface. 

A detector accorcfing to Claim 1 wherein said warn- 
ing mechanism comprises a display for displaying 
the color intage provkled k>y said video camera with 
the area of interest delineated and the foreign 
object highlighted. 

A detector accorcfing to Claim 1 wherein said warn- 
ing mechanism comprises a light for illumination 
upon detecting a foreign object 

A detector according to Claim 1 wherein said warn- 
ing mechanism comprises an audible warning 
device fbr emitting an audfo signal upon detecting a 
foreign object. 

A detector according to Claim 1 further comprising 
an optical pointing device for receiving the location 
off the foreign object from said color image process- 
ing system and for specifically illuminating the for- 
eign object on the work surface. 

A detector according to Claim 1 wherein said color 
image processing system discriminates tiie foreign 
object based upon the colors of tiie objects in the 
area of interest of the color image. 

A detector according to Claim 6 wherein said color 
image processing system further comprises means 
for accepting a list of colors associated witti foreign 
objects. 

A detector according to Claim 6 wherein said color 
image processing system discriminates the foreign 
abject by detecting objects having a color different 
from the color of the background in the area of inter- 
est of tfie digital color image. 

A detector according to Claim 8 wherein said color 
image processing system further comprises means 
tor ignoring one or more predetermined colors so 
as to reduce false alarms. 
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10. A detector according to Claim 8 wherein said color 
image processing system further comprises means 
for redudng illumination glare on the wor1< surface 
so as to reduce false alarms. 

11. A detector according to Claim 8 wherein said color 
image processing system reduces false alarms by 
further comprising means for subtracting a prede- 
fined representation of the field of view from the 
color image so that remaining discreparides repre- 
sent foreign objects. 

12. A detector according to Claim 1 wherein said color 
image processing system further comprises 
means, responsive to operator actuation, for 
accepting the definition of the area of interest 

13. A detector according to Claim 1 wherein said color 
image processing system further comprises means 
for accepting the definition of the area of interest in 
the form of an electronic computer automated 
design file. 



14. A method for detecting a foreign object during man- 
ufacturing of products that indude one or more lay- 
ers of material and for warning an operator of the 
detected foreign object, said process comprising 
the steps of : 

obtaining a video representatbn of a work sur- 
face; 

transmitting the video representation as a 
video signal: 

converting the video signal to a digital color 

image; 

sut>tracting predefined areas in which the pres- 
ence of foreign objects is unimportant from the 
digital color image; 

detecting a foreign object based upon the color 
of the foreign object; and 
warning the operator of the preserK^e of the for- 
eign object to permit the foreign object to be 
removed before any further manufecture of 
product. 

15. A method according to Claim 14 wherein said step 
of warning the operator of the presence of the for- 
eign object so that the foreign object may be 
removed further comprises displaying the color 
image and highlighting the foreign otsject within the 
displayed color imaga 
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16. A method according to Claim 14 wherein said step 
of subtracting predefined areas from the color 
image comprises the steps of: ss 

retaining a portion of the color image that cor- 
responds to portions of the work surface con- 



taining the one or more layers of material; 
sut>tracting colors of the one or more layers of 
material from the retained portion of the cdor 
image; arxf 

designating those objects within the retained 
portion of the color image wh^ remain 
colored as foreign ot)jects. 

17. A method according to Claim 16 wherein said step 
of sulJtnacting predefined areas from the color 
image further comprises the step of subtracting var- 
iations in the digital color image attributatsle to illu- 
mination glare on the work surface from the digital 
ook>r Image. 

ia A method according to Claim 14 wherein said step 
of warning the operator of the presence of the for- 
eign object so tiiat tiie foreign ot^ect may be 
renrx>ved further comprises illuminating a warning 
20 light. 

19. A method according to Claim 14 wherein said step 
of warning the operator of the presence of the for- 
eign object so that tiie foreign object may ble 
removed furtiier conprises transmitting an audio 
signal. 

2a The method of Claim 14 in which said step of 
obtaining a video representation of the work sur- 
face further conrprises the step of obtaining video 
representations of the work surface from a plurality 
of vantage points to increase the resolution of tiie 
resulting digital cdor image, thereby fadlrtating 
detection of foreign objects while permitting addi- 
tional contour variatior^ in the work surfece. 

21. The detector of Claim 6 in which said color image 
processing system further comprises a neural net- 
work trained to recognize foreign objects. 

22. A detector accorc£ng to Claim 1 wherein said warn- 
ing mechanism provides an electromagnetic signal. 

23. A detector for finding foreign objects laying on a 
work surfoce and providing warning, the detector 
comprising: 

a video camera having a field of view, said 
video camera oriented such tiiat the work sur- 
face is in the field of view for producing a vkfeo 
signal thereof; 

a converter, responsive to said video camera, 
for iteratively transforming the video signal to a 
digital color inrrage; 

a video signal carrier extending between saki 
video camera and said converter for transfer- 
ring the video signal to the converter; 
a color Image processing system, responsive 
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to said video camera, for analyzing a prede- 
fined area of interest within the field of view of 
the video camera, said color image processing 
system corrprising means for discriminating a 
foreign object witfiin the area of interest of the 5 
color image based on visual attributes of the 
foreign object, said color image processing 
system further comprising means for determin- 
ing a location of the foreign object on the work 
surbce: and ;o 
a warning mechanism, responsive to said color 
image processing system, for an alerting the 
operator when a foreign object is present on 
the work surface. 
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